The present invention relates generally to data communication systems and methods and, more particularly, to data communication systems and methods in which memory-mapped receive and transmit buffers are provided to network interface controllers.
A network interface controller (NIC) is a hardware device that supports the transmission of data between computers as illustrated in FIG. 1. Therein, a symmetric multiprocessor (SMP) system 10 includes a number of central processor units (CPUs) 12 which share memory unit 14 via a memory interconnect 16. Although SMP 10 is shown as having four CPUs (cores), those skilled in the art will appreciate that SMP 10 can have more or fewer CPUs. SMP 10 sends messages to other SMPs 20, 30 and 40 under the control of NIC 18 via Ethernet connections and a fabric (switch) 22. The NIC 18 will typically have a processor (not shown) associated therewith, either as an integral part of the NIC or in the form of a helper processor, so that the NIC has sufficient intelligence to interpret various commands. The NIC 18, as well as various I/O devices 42, 44, are connected to the rest of the SMP via an I/O interconnect 46. The I/O interconnect 46 communicates with the memory interconnect 16 via an I/O adapter 48 (e.g., a bridge).
A common source and destination for transmitted data in such systems is paged virtual memory. Paged virtual memory provides for virtual addresses which are translated or mapped onto physical pages and enables virtual pages to be swapped out to disk or removed from main memory and later swapped in from disk to a new physical page location. An operating system can unilaterally perform page swaps of so-called “unpinned” virtual pages. Thus, application software operating on such systems typically accesses main memory using address translation hardware that ensures that the correct physical page is accessed, e.g., that the operating system has not initiated a page swap for the page that the application software needs to access. Software access pauses during time intervals when needed data is swapped out and resumes by accessing a new physical location when data is swapped in at that location.
Some networking solutions address the downtime associated with software suspension during virtual page swapping by providing for software to copy data from unpinned virtual memory to pinned interface memory. Pinned memory consists of pages that cannot be swapped to disk by the operating system. In such systems, the NIC 18 will typically access only pinned interface memory. This simplifies direct memory access (DMA) transfers performed by the NIC 18, since data is never swapped during a network operation which, in turn, guarantees that data remains accessible throughout a NIC's DMA data transfer and that the physical address of the data remains constant. However, such solutions require extra overhead in the form of data copying (e.g., copying from unpinned virtual memory to a pinned system buffer accessible by the NIC 18) that utilizes important system resources.
Another solution to the issue posed by unpinned virtual memory eliminates the above-described data copying but instead requires that the NIC 18 invoke an operating system function to pin a virtual page prior to transmitting data directly from or to that page. Additionally, the page must later be unpinned by a further NIC/operating system interaction in order to allow page swapping after network activity is finished. While this eliminates copies to pinned pages, the NIC 18 must now invoke expensive page pinning and page unpinning functions. Each of these operations requires communication between the NIC's processor and the operating system. When these communications require interrupts or polling of the I/O interconnect 46, they are very expensive in terms of resource utilization efficiency.
Accordingly, it would be desirable to provide mechanisms and methods which enable a NIC to more efficiently deal with data transfer issues.